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DESCRIPTION 

REMOTE CONTROL SYSTEM, REMOTE COMMANDER, AND REMOTE 

CONTROL SERVER 

Technical Field 
[0001] 

The present invention relates to a remote control 
system, a remote commander, and a remote control server 
for remotely operating a television receiver, a DVD 
player, and various other devices, such as an AV device, 
a CE device, and an information device, by using a remote 
commander, and more particularly, to a remote control 
system, a remote commander, and a remote control server 
for realizing remote control free from limitations on the 
directivity or available transmission distance from the 
remote commander to a device to be operated. 
[0002] 

More specifically, the present invention relates to a 
remote control system, a remote commander, and a remote 
control server for providing two-way communication 
between the remote commander and a device to be operated 
using a networlc such as TCP/IP, and more particularly, to 
a remote control system, a remote commander, and a remote 
control server for remotely controlling a device in front 
of the user's eyes as a device to be operated from a 
plurality of devices without limitations on the 
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directivity or available transmission distance. 
Background Art 
[0003] 

To date, various information appliances and home 
electric appliances, such as television receivers, video 
recording and playback devices, and audio- visual devices, 
have been developed and manufactured, and have been 
widely used in houses and other living spaces. Basically, 
such information devices are directly operated through 
user interfaces generally provided for the device main 
body, such as operation buttons and volume switches. 
Recently, however, almost all devices can be remotely 
operated using remote controllers. 
[0004] 

For example, in the field of consumer electrical and 
electronic devices, such as home electrical products, 
"infrared remote controllers" using an AM modulation 
method as means for inputting user operation commands to 
the devices in a remote manner have been developed and 
have already been established. 
[0005] 

Such a communication method using infrared light has 
advantages of low cost, low power consumption, 
substantially no legal restrictions in countries, and so 
forth. Elimination of hard-wired cabling also means that 
connectors for connecting cables are not necessary, 
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leading to reduction in cost. There is no concern of 
mechanical exhaustion caused by inserting and removing 
connectors each time connection or disconnection occurs. 
[0006] 

The communication method using infrared light, 
however, has a problem of directivity. Communication is 
not established unless a light-receiving unit of a 
receiver is set within the angle of view of a .transmitter, 
which imposes a large limitation in use (see, for example. 
Patent Document 1) . Further, the infrared communication 
method basically provides one-way communication, and has 
a problem that a transmission source is not able to 
receive a response from the other party and is not able 
to make a delivery confirmation.- 
[0007] 

It is possible to carry out two-way communication if 
both devices that are to perform communication are 
provided with an infrared transmission function and an 
infrared reception function. In infrared communication, 
however, due to the problem of directivity, users of both 
devices are inconvenienced by having to direct their 
devices to the other devices to perform transmission 
operations. 
[0008] 

For example, a communication device that relays an 
infrared operation signal for an electronic device, which 
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is emitted from a remote commander, to operate an 
electronic device located at a place where an infrared 
operation signal from a dedicated remote controller does 
not directly reach has been proposed (see, for example, 
Patent Document 2) . This communication device extracts a 
pulse waveform component by removing a carrier component 
from the infrared operation signal received from the 
remote commander, and transmits the pulse waveform 
component through a network. The communication device 
also modulates an infrared carrier by the pulse waveform 
component, and transmits the modulated infrared carrier 
to the electronic device to be operated. 
[0009] 

A remote operation system that is constructed with a 
simple device configuration in which no relay device or 
the like is used has been proposed (see, for example, 
Patent Document 3) . A controller transmits capability 
information indicating the capability of input operations 
that can be performed by the controller to a device to be 
controlled, and the device to be controlled creates and 
returns remote-controller handling information for 
forming a user interface that can be realized within the 
range of the received capability information. The 
controller forms a user interface using the remote- 
controller handling information. 
[0010] 
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In the infrared communication method, however, due to 
the limited communication bandwidth, it is difficult to 
exchange a large amount of data such as moving pictures 
or still images, and the function of a terminal device 
for performing remote control is limited. 
[0011] 

Further, a mobile-phone remote control system for 
allowing a reservation to be set in devices, such as home 
electric appliances, by using a mobile phone has been 
proposed (see, for example. Patent Document 4) . 
According to this system, a remote controller receives a 
signal from a mobile phone, and optically transmits a 
remote control signal for remotely controlling a device 
to be controlled according to the received signal. 
[0012] 

In the remote control system using a network, a 
mobile telephone network, or the like, a remote operation 
itself is possible; however, limitations on the 
communication bandwidth are still imposed, as in a 
standard remote control system, because the infrared 
communication method is adopted in the final stage in 
remote control . 
[0013] 

[Patent Document 1] 

Japanese Unexamined Patent Application Publication No. 
2002-165281 
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[Patent Document 2] 

Japanese Unexamined Patent Application Publication No. 

2003-258464 

[Patent Document 3] 

Japanese Unexamined Patent Application Publication No. 

2003-143670 

[Patent Document 4] 

Japanese Unexamined Patent Application Publication No. 

11-284757 

Disclosure of Invention 

Problems to be Solved by the Invention 

[0014] 

It is an object of the present invention to provide a 
superior remote control system, a superior remote 
commander, and a superior remote control server, in which 
a television receiver, a DVD player, and various other 
devices, such as an AV device, a CE device, and an 
information device, can suitably be operated remotely by 
using a remote commander. 
[0015] 

It is another object of the present invention to 
provide a superior remote control system, a superior 
remote commander, and a superior remote control server, 
in which remote control free from limitations on the 
directivity or available transmission distance from the 
remote commander to a device to be operated can be 
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realized. 

[0016] 

It is another object of the present invention to 
provide a superior remote control system, a superior 
remote commander, and a superior remote control server, 
in which two-way communication between the remote 
commander and a device to be operated can be carried out 
using a network such as TCP/IP. 
[0017] 

It is another object of the present invention to 
provide a superior remote control system, a superior 
remote commander, and a superior remote control server, 
in which a device in front of the user's eyes can be 
specified as a device to be operated from a plurality of 
devices without limitations on the directivity or 
available transmission distance and can be remotely 
controlled. 

Means for Solving the Problems 
[0018] 

The present invention has been made in view of the 
above-mentioned programs, and provides a remote control 
system in which a remote commander is used to operate one 
or more devices to be controlled, wherein the remote 
commander and at least some of the devices to be 
controlled are provided with a network communication 
function, and a device to be controlled is operated by 
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means of a command through a network in response to a 
user input on the remote commander. 
[0019] 

The term "system" as used herein refers to a logical 
collection of a plurality of apparatuses (or function 
modules achieving a specific function) regardless of 
whether or not the apparatuses or function modules are 
housed in a single housing (the same applies to the 

following description) . 

[0020] 

In the remote control system according to the present 
invention, the device to be controlled returns a response 
through the network in response to a command transmitted 
by the remote commander through the network. 
[0021] 

Further, the remote commander submits a request for 
data to the device to be controlled, and the device to be 
controlled returns the requested. data through the network. 
Then, the remote commander decodes the data received from 
the device to be controlled to play back and output the 
data. 
[0022] 

To date, a communication method using infrared light 
has been widely used to remotely operate devices". The 
communication method using infrared light has advantages 
of low cost, low power consumption, substantially no 
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legal restrictions in countries, and so forth. However, 
the communication method using infrared light has a 
problem of directivity. Communication is not established 
unless a light-receiving unit of a receiver is set within 
the angle of view of a transmitter, which imposes a large 
limitation in use. Further, the infrared communication 
method basically provides one-way communication, and a 
transmission source is not able to receive a response 
from the other party and is not able to make a delivery 
confirmation . 
[0023] 

In the remote control system according to the present 
invention, in contrast, a communication medium with fewer 
limitations on the directivity or available communication 
range, such as an IP network, is used to perform a remote 
control operation. 
[0024] 

According to such a remote control system using an IP 
network, two-way commxinication can be carried out between 
a remote controller and a device to be operated, and 
teclmical advantages of providing more reliable 
communication by means of a delivery confirmation 
(response) , handling a GUI -based complex command scheme, 
and providing large -volume data transmission, such as 
moving- image streaming, using relatively broad bands are 
achieved. For example, a television receiver to be • 
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operated by the remote controller distributes moving- 
image data subjected to reception processing, such as a 
child view, to an IP remote controller through an IP 
network so that the child view can be viewed on a display 
screen of the. IP remote controller. 
[0025] 

When data is requested by the remote commander, the 
device to be controlled may convert the corresponding 
data content into a format that can be played back and 
output by the remote commander, and returns the converted 
data through the network. For example, when a request 
for video output of a desired television program or a 
request for AV output video from a device to be 
controlled, such as an HDD recorder, is submitted to a 
television receiver serving as a device to be controlled, 
the video data is converted into a low-bit -rate data 
format that can be received and played back by an IP 
remote controller, such as MPEG4 data, and is distributed 
via streaming through a LAN. However, the process for 
conversion into a data format that can be played back and 
output by the remote commander according to the present 
invention is not limited to conversion of bit rates, such 
as conversion from a high bit rate to a low bit rate. 
[0026] 

Devices to be controlled may include an IR device 
that can be remotely controlled only by means of old 
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infrared communication. In this case, a remote control 
server having a network communication function and a 
protocol conversion function of converting a command 
received through a network into an infrared command may- 
be provided between the IP remote controller and the IR 
device. Upon receiving an operation command for the IR 
device from the IP remote controller through an IP 
network, the remote control server converts the operation 
command into an infrared command, and transfers the 
infrared command to the IR device. Therefore, the IR 
device can be operated by the IP remote controller. 
[0027] 

The remote control server may extract from the IR 
device the data requested by the remote commander to the 
IR device, and may convert the extracted data into a 
format that can be played back and output by the remote 
commander to return the converted data to the remote 
commander through the network. For example, the remote 
control server converts AV output video from a recording 
device serving as an IR device into a low-bit- rate data 
format that can be received and played back by the IP 
remote controller,' such as MPEG4 data, and distributes 
the converted data via streaming through a LAN. 
[0028] 

If the device to be controlled is a television 
receiver, it can extract EPG data from a received 
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broadcast wave and can store the EPG data. In this case, 
the EPG data may be returned through the network in 
response to an EPG data request from the remote commander. 
The remote commander displays and outputs received EPG 
data. 
[0029] . 

The remote commander may transmit a channel change 
request to a television receiver serving as a device to 
be controlled in response to designation of a channel on 
a current EPG data display view. 
[0030] 

In response to this, the television receiver may 
convert video content received on the channel specified 
in the change request given by the remote commander into 
a format that can be played back and output by the remote 
commander, and may distribute the converted video content 
via streaming through the network. Then, the remote 
commander decodes the video content received, and 
displays video. The user can view a child view on the 
remote commander. 
[0031] 

Further, the television receiver may change the 
display of the video to the channel specified in the 
change request given by the remote commander. This is an 
operation of throwing the video displayed on the remote 
commander onto a television screen. 
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[0032] 

Further, in response to designation of a television 
program on a future EPG data display view, the remote 
commander may transmit a request for setting a 
reservation to record the program to a recording device 
serving as a- device to be controlled. 
[0033] 

A recording reservation from the remote commander to 
the recording device may be set by the intervention of a 
remote control server. Upon receiving a recording 
reservation request from the remote commander through the 
network, the remote control server converts the recording 
reservation request into infrared reservation data on the • 
basis of EPG data, and transmits the recording 
reservation data via infrared light to an IR recording 
device that can be remotely controlled only by means of 
infrared communication. Then, the IR recording device 
sets a recording reservation according to the infrared 
reservation data from the remote control server. 
[0034] 

Further, in response to a request for video content 
form the remote commander, a recording device serving as 
a device to be controlled may convert the requested video 
content into a format that can be played back and output 
by the remote commander, and may distribute the converted 
video content via streaming through the network. The 
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remote commander decodes the video content received, and 
displays video. The user can confirm the recorded video 
on the remote commander without changing the video 
displayed on the television set. 
[0035] 

Such an operation for recorded video on the remote 
commander may be performed by the intervention of a 
remote control server. Upon receiving a video content 
request from the remote commander through the network, 
the remote control server converts the video content 
request into an infrared command, and transmits the video 
content request via infrared light to an IR recording 
device that can be remotely controlled only by means of 
infrared communication. In response to this, the IR 
recording device outputs video content according to the 
infrared command from the remote control server. Then, 
the remote control server converts the video content 
output from the IR recording device into a format that 
can be played back and output by the remote commander, 
and distributes the converted video content to the remote 
commander via streaming through the network. 
[0036] 

The remote commander can request a display device 
serving as a device to be controlled to change the 
display of the video to video content currently being 
displayed on the remote commander. In response to the 
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request to change the display of the video, for example, 
the display device changes the screen from video received 
by a television set to video output from a recording 
device. This is an operation of throwing the video 
displayed on the remote commander onto a television 
screen. 
[0037] 

Further, the remote commander may submit a request 
for device information to devices to be controlled having 
a network communication function through the network at 
the turn-on time or any other timing. Then, a device 
list concerning a device to be controlled that has 
responded with device information in response to the 
request is displayed. A device to be controlled that 
responded with device information in the past and that 
does not respond at present may be displayed in a grayed- 
out manner on the device list. Alternatively, a device 
to be controlled that has not responded with device 
information for a certain period of time or more may be 
deleted from the device list. 
[0038] 

The remote commander can transmit an operation 
request to a device to be controlled selected on the 
device list. When the device to be controlled, displayed 
in a grayed-out manner on the device list, is selected, 
an operation request may be transmitted after a request 
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for turning on the device to be controlled is submitted. 
[0039] 

Further, the remote control server may pre-register 
therein information on IR devices to which the infrared 
command can be transmitted, and may return IR device 
information to the remote commander in response to a 
request from the remote commander. 
[0040] 

In this case, the remote commander displays an IR 
device list. Then, the remote commander can transmit an 
operation request for an IR device selected on the IR 
device list to the remote control server through the 
network. In response to this, the remote control server 
converts the operation request for the IR device from the 
remote commander into an infrared command, and transfers 
the converted infrared command to the IR device. 
Advantages 
[0041] 

According to the present invention, a superior remote 
control system, remote control method, remote commander, 
and electronic device in which a television receiver, a 
DVD player, and various other devices, such as an AV 
device, a CE device, and an information device, can 
suitably be operated remotely by using a remote commander 
can be provided. 
[0042] 
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Further, according to the present invention, a 
superior remote control system, remote commander, and 
remote control server in which remote control free from 
limitations on the directivity or available transmission 
distance from a remote commander to a device to be 
operated can be realized. 
[0043] 

Further, according to the present invention, a 
superior remote control system, remote commander, and 
remote control server in which two-way communication 
between the remote commander and a device to be operated 
can be carried out using a network, such as TCP/IP, can 
be provided. 
[0044] 

Further, according to the present invention, a 
superior remote control system, remote commander, and 
remote control server in which a device in front of the 
user's eyes can be specified as a device to be operated 
from a plurality of devices without limitations on the 
directivity or available transmission distance and can be 
remotely controlled can be provided. 
[0045] 

Other objects, features, and advantages of the 
present invention will become apparent from the following 
embodiments of the present invention and more detailed 
description with reference to the accompanying drawings. 
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Brief Description of the Drawings 
[0046] 

Fig. 1 is a diagram schematically showing an example 
structure of a remote control system according to an 
embodiment of the present invention. 

Fig. 2 is a diagram showing a remote control system 
according to another embodiment of the present invention. 

Fig. 3 is a diagram showing an example hardware 
configuration of a terminal device capable of operating 
as an IP remote controller. 

Fig. 4 is a diagram, schematically showing a structure 
of an apparatus operating as a remote control server. 

Fig. 5 is a diagram illustrating a manner in which 
EPG (current) data received by a television set is 
obtained by an IP remote controller and is displayed an 
LCD 19. 

Fig. 6 is a diagram illustrating a manner in which a 
television channel change operation is carried out 
through an EPG (current) view displayed on the IP remote 
controller. 

Fig. 7 is a diagram illustrating a manner in which 
EPG (future) data received by the television set is 
obtained by the IP remote controller and is displayed on 
the LCD 19. 

Fig. 8 is a diagram illustrating a manner in which a 
recording reservation of a program is set through an EPG 
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(future) view displayed on the IP remote controller. 

Fig. 9 is a diagram showing an operation sequence for 
displaying the EPG (current) data on the IP remote 
controller. 

Fig. 10 is a flowchart showing a processing procedure 
executed by the IP remote controller to display the EPG 
(current) data. 

Fig. 11 is a diagram showing an example of an 
operation sequence in which a recording reservation of a 
program is set in a recording device by the IP remote 
controller using the EPG (future) data. 

Fig. 12 is a diagram showing another example of the 
operation sequence in which a recording reservation of a 
program is set in the recording device by the IP remote 
controller using the EPG (future) data. 

Fig. 13 is a diagram showing still another example of 
the operation sequence in which a recording reservation 
of a program is set in the recording device by the IP 
remote controller using the EPG (future) data. 

Fig. 14 is a diagram illustrating a manner in which a 
terrestrial analog broadcast received by the television 
set is viewed on the IP remote controller. 

Fig. 15 is a diagram illustrating a manner in which a 
channel is changed in a period of time during which a 
terrestrial analog broadcast received by the television 
set is viewed on the IP remote controller. 
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Fig. 16 is a diagram illustrating a manner in which a 
channel is changed in a period of time during which the 
terrestrial analog broadcast received by the television 
set is viewed on the IP remote controller. 

Fig. 17 is a diagram illustrating a manner in which a 
channel on the television set is changed to a television 
program currently being viewed on the IP remote 
controller. 

Fig. 18 is a diagram illustrating a manner in which a 
mode in which recorded content stored in a recording 
device is currently viewed on the IP remote controller is 
changed so that the recorded content is also viewed on 
the television set. 

Fig. 19 is a diagram illustrating a manner in which 
the recorded content stored in the recording device is 
viewed on the IP remote controller. 

Fig. 20 is a diagram showing an example of an 
operation sequence for streaming AV content from an IR 
recording device to the IP remote controller. 

Fig. 21 is a diagram showing an example of an 
operation sequence for streaming AV content from a DLNA 
device to the IP remote controller. 

Fig. 22 is a diagram showing an example of an 
operation sequence for throwing content of an IR device 
currently being viewed on the IP remote controller onto 
the television set . 
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Fig. 23 is a diagram showing an example of an 
operation sequence for throwing content of a DLNA device 
currently being viewed on the IP remote controller onto 
the television set. 
Reference Numerals 
[0047] 



11 


CPU 


12 


bus 


13 


memory device 


14 


input switch matrix unit 


15 


network communication unit 


16 


infrared communication unit 


17 


codec processing unit 


18 


graphic display processor 


19 


liquid crystal display unit 


20 


data input /output interface 


31 


CPU 


32 


bus 


33 


memory device 


34 


wired LAN unit 


35 


infrared processing unit 


37 


codec processing unit 


38 


HDD 



Best Mode for Carrying Out the Invention 
[0048] 

An embodiment of the present invention will be 
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described in detail hereinafter with reference to the 
drawings . 
[0049] 

The present invention relates to a remote control 
system in which a television receiver, a DVD player, and 
various other devices, such as an AV device, a CE device, 
and an information device, are remotely operated by using 
a remote commander. 
[0050] 

A communication method using infrared light, that is, 
an infrared remote controller, is generally used for 
remote control of devices. However, due to the problem 
of directivity, communication is not established unless a 
light -receiving unit of a receiver is set within the 
angle of view of a transmitter, which imposes a large 
limitation in use. Further, the infrared communication 
method basically provides one-way communication, and a 
transmission source is not able to receive a response 
from the other party, and is not able to make a delivery 
confirmation. 
[0051] 

In the present invention, therefore, a remote control 
system using a remote commander based on an IP network 
instead of an infrared transmission path, i.e., an IP 
remote controller, is constructed. The IP remote 
controller provides remote control using a communication 
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medium with fewer constraints on the directivity or 
available communication range. Further, on an IP network, 
two-way communication can be carried out between the 
remote controller and a device to be operated, and 
technical advantages of providing more reliable 
communication by means of a delivery confirmation 
(response) , handling a GUI -based complex command scheme, 
and providing large -volume data transmission, such as 
moving- image streaming, using relatively broad bands are 
achieved. For example, a television receiver to be 
operated by a remote controller distributes moving- image 
data subjected to reception processing, such as a child 
view, to an IP remote controller through an IP network so 
that the child view can be viewed on a display screen of 
the IP remote controller. 
[0052] 

Since an IP remote controller for controlling a 
device on a network is non-directional, or permeable, 
there arises a problem that a particular device in front 
of the user's eyes cannot be designated and operated as a 
target. On account of this, remote control is performed 
using a TCP/IP network. An infrared remote control 
function of the related art may also be used to specify a 
device. 
[0053] 

A. System Configuration 
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Fig. 1 schematically shows an example structure of a 
remote control system according to an embodiment of the 
present invention. The system shown in Fig. 1 includes 
an IP remote controller used by a user to operate devices, 
and controlled devices A and B to be operated by the IP 
remote controller. The controlled devices A and B are 
located in different rooms 1 and 2, respectively, and 
both devices do not simultaneously reside in an area 
where infrared light reaches. 
[0054] 

The controlled devices A and B include, for example, 
a television receiver, a DVD player, and various other 
devices, such as an AV device, a CE device, and an 
information device, and are each provided with a function 
of receiving an infrared command and a network interface. 
The controlled devices A and B are connected to each 
other via an IP network (LAN) established under IEEE 
802.3 (ETHERNET™ (registered trademark)) or the like. 
[0055] 

An example of controlled devices that can be operated 
by means of commands from the IP remote controller 
through a network is a home electric appliance compatible 
with the DLNA (Digital Living Network Alliance) 
guidelines, and is hereinafter also referred to as a 
"DLNA device" . 
[0056] 
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The IP remote controller has a function of 
transmitting an infrared command in accordance with a 
user's operation and a wireless network interface, and is 
capable of communicating with an access point (AP) 
through a wireless network established under IEEE 
802.11a/b or the like. Since the AP is connected to the 
IP network, the IP remote controller can perform two-way 
communication with the controlled devices A and B over 
the AP to transmit a remote control command or receive 
data from the controlled devices A and B through the IP 
network . 
[0057] 

The IP remote controller may further include a 
display, such as an LCD, and may play back and output 
moving- image data distributed via streaming from the 
controlled devices A and B through the IP network 

(discussed below) . 

[0058] 

Fig. 2 shows a remote control system according to 
another embodiment of the present invention. A major 
difference from the system configuration shown in Fig. 1 
is the intervention of a remote control server between an 
IP remote controller and a device to be controlled. 
[0059] 

The remote control server is used by connecting it to 
a main DLNA device to be controlled by the IP remote 
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controller, such as a television receiver, or, 
alternatively, is integrally formed with this type of 
DLNA device. 
[0060] 

The remote control server has a wired LAN 
communication function, such as ETHERNET™, and receives a 
command from the IP remote controller through the IP 
network, i.e., the LAN, to transfer an operation command 
to a device to be controlled by the IP remote controller. 
If the device to be controlled also has a LAN 
communication function, the remote control server 
transfers . through the LAN an operation command received 
from the IP remote controller. 
[0061] 

Further, for the benefit of a device to be controlled 
without a LAN communication function, which can be 
remotely operated only by an old (legacy) infrared (IR) 
remote control method (the device to be controlled is 
hereinafter also referred to as an "IR device"), the 
remote control server converts an operation command 
received from the IP remote controller through the LAN 
into a command format for use in infrared communication 
such as SIRCS, and performs optical transmission. For 
transmission of an infrared command, if the IR device to 
be controlled is in a dead angle from the remote control 
server or the distance therebetween is long, an extension 
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cord, such as an AV mouse, may be used to optically 
transmit the infrared command in the vicinity of the IR 
device. The AV mouse is disclosed in, for example, 
Japanese Unexamined Patent Application Publication No. 
2001-223955, which has been assigned to the present 
applicant. 
[0062] 

When the remote control server is connected to a 
television receiver or any other source device of AV 
content or is integrally formed therewith, the remote 
control server can also function as a distribution server 
that distributes the AV content (or any other form of 
content) to the IP remote controller. 
[0063] 

For example, when the remote control server is 
connected to a television receiver, received audio and 
video signals are subjected to AV codec conversion into a 
low-bit -rate data format that can be received and played 
baclc by the IP remote controller, such as MPEG4 data, and 
are then distributed to the IP remote controller via 
streaming through the LAN. Alternatively, recorded 
content recorded at a high bit rate, such as MPEGl or 
MPEG2 content, is loaded from an HDD recorder connected 
through the LAN, is subjected to AV codec conversion into 
a low-bit -rate data format that can be received and 
played baclc by the IP remote controller, such as MPEG4 
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data, and is then distributed to the IP remote controller 
via streaming through the LAN. Conversion of bit rates, 
such as, but not limited to, conversion from a high bit 
rate to a low bit rate, is an example of the process for 
conversion into a data format that can be played back and 
output by the IP remote controller. 
[0064] 

Fig. 3 shows an example hardware configuration of a 
terminal device capable of operating as the IP remote 
controller in the remote control system according to the 
present embodiment. The terminal device shown in Fig. 3 
may be manufactured as a dedicated terminal for the IP 
remote controller, or can be designed to also function as 
another portable terminal, such as a PDA (Personal 
Digital Assistant) or a game device. 
[0065] 

An IP remote controller 10 shown in Fig. 3 is 
configured such that a CPU (Central Processing Unit) 11 
generally controls the respective parts via a bus 12 . 
[0066] 

The CPU 11 is provided with a memory device 13 
including a ROM and a RAM, and loads program code stored 
in the ROM onto the RAM to perform a predetermined 
process. The predetermined process includes operations, 
such as transmission and reception of a command/ response 
of a control signal to and from a device to be controlled 
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through a network, transmission and reception of content 
to and from the device to be controlled, file transfer, 
and remote operation of a device to be controlled, such 
as a DLNA device or an IR device, and these operations 
are executed according to an operation input from a user. 
The details of the operations are discussed below. 
[0067] 

An input switch matrix unit 14 includes a key 
operation unit including keys, such as a ten-key pad, an 
audio adjustment key, an image quality adjustment key, 
and a channel selection key, and is operated to input the 
contents of an operation to be performed on a device to 
be remotely controlled by a user. A controlled- device 
operating command input by the input switch matrix unit 
14 is transmitted from a network communication unit 15 
through an IP network. 
[0068] 

The network communication unit 15 is provided with a 
network interface for communicating with an access point 
(AP) through a wireless network established under, for 
example, IEEE 802.11a/b or the like. The network 
communication unit 15 is assigned identification 
information unique on the network, such as a MAC address 
or an IP address. In the present embodiment, the network 
communication unit 15 is configured to transfer a remote 
control request in accordance with an instruction given 
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by the input switch matrix unit 14 to a device to be 
controlled through the network. 
[0069] 

An infrared communication unit 16 is configured to 
transmit as an infrared signal a device search request 
for specifying a device in front of the eyes of the user 
who is an owner of the IP remote controller. The 
infrared communication unit 16 may also be provided with 
a function for transmitting a standard infrared command 
with the use of SIRCS (Serial Infrared Remote Control 
System) , which is an infrared communication standard 
widely used for remote controllers for AV devices and so 
forth. 
[0070] 

A codec processing unit 17 performs encoding and 
decoding of AV content. In the present embodiment, AV 
content received by the network communication unit 15 
from a device to be controlled, such as MPEG4 content, 
can be decoded to play back and output audio and video. 
For example, a television receiver to be operated by the 
remote controller distributes moving- image data subjected 
to reception processing, such as a child view, to the IP 
remote controller through the IP network so that the 
child view can be viewed on a display screen of the IP 
remote controller. In a case where AV content is not 
transmitted from the IP remote controller, the encoding 
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function of the codec processing unit 17 is not essential. 
[0071] 

A graphic display processor (GDP) 18 processes 
graphic data to be displayed and outputted to a liquid 
crystal display unit (LCD) 19, and controls the driving 
of the LCD 19. For example, a video signal decoded by 
the codec processing unit 17 is displayed and output from 
a screen of the LCD 19 by the GDP 18. Further, based on 
device information obtained from a device to be 
controlled connected to the network, a user interface for 
operating the device to be controlled is displayed and 
output on the LCD 19. 
[0072] 

The IP remote controller further includes a data 
input/output interface 20 formed of a wired interface, 
such as USB (Universal Serial Bus) , a slot for a memory 
card, or the like. AV content transferred through the 
USB interface or AV content stored in the memory card can 
be decoded by the codec processing unit 17 to play back 
and output data. Alternatively, data processed on the IP 
remote controller can be encoded by the codec processing 
unit 17 to transfer the data from the USB interface or 
write it in the memory card. 
[0073] 

Fig. 4 schematically shows a structure of an 
apparatus operating as the remote control server in the 
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remote control system according to the present embodiment . 
The remote control server is used by connecting it to a 
main AV device to be controlled by the IP remote 
controller, such as a television receiver, or, 
alternatively, is integrally formed with this type of AV 
device . 
[0074] 

A remote control server 30 shown in Fig. 4 is 
configured such that a CPU 31 generally controls the 
respective parts via a bus 32. 
[0075] 

The CPU 31 is provided with a memory device 33 
including a ROM and a RAM, and loads program code stored 
in the ROM onto the RAM to perform a predetermined 
process . 
[0076] 

The remote control server 3 0 includes a wired LAN 
communication unit 34, such as ETHERNET"^^, and receives a 
command from the IP remote controller through the LAN and 
the AP to perform processing on the operation command. 
[0077] 

The remote control server 30 further includes an 
infrared processing unit 35. For the benefit of an IR 
device that can be operated only by an old infrared 
remote control method, the infrared processing unit 35 
converts an operation command received from the IP remote 
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controller through the LAN into a command format for use 
in infrared communication such as SIRCS, and performs 
optical transmission by means of infrared light. If the 
legacy controlled device is in a dead angle from the 
remote control server or the distance therebetween is 
long, an infrared command is output to an AV mouse so 
that the infrared command is optically transmitted in the 
vicinity of the legacy controlled device through output. 
[0078] 

A codec processing unit 37 performs encoding and 
decoding of AV content. In the present embodiment, the 
codec processing unit 37 is provided with input terminals 
for audio and video signals, and encodes audio and video 
signals input from a device to be controlled, such as a 
television receiver or an analog video recorder, into a 
data format that can be played back on the IP remote 
controller, such as MPEG4 data, to distribute the encoded 
data to the IP remote controller through the IP network. 
Alternatively, the codec processing unit 37 loads 
recorded content recorded at a high bit rate, such as 
MPEGl or MPEG2 content, from an HDD recorder serving as a 
device to be controlled through the LAN, converts it into 
a low-bit -rate data format that can be received and 
played back by the IP remote controller, such as MPEG4 
data, and distributes the converted data via streaming 
through the LAN. 
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[0079] 

The remote control server 3 0 may further include a 
large- capacity storage device 38, such as an HDD, for 
storing encoded or decoded AV content, EPG data separated 
from a received broadcast wave, and a large volume of 
other data. However, the HDD 3 8 is not essential to the 
remote control server 30. 
[0080] 

B. Functionality of Remote Control System 

According to a remote control system using an IP 
remote controller, by connecting the IP remote controller 
and a device to be controlled through a networlc, 
communication operations, such as communication of a 
control signal and content and file transfer, can be 
performed between the IP remote controller and the device 
to be controlled. These communication operations are 
performed, thereby achieving the following basic 
functions : 
[0081] 

(1) Control signal (command/ response) 

To provide an interactive operation environment, such 
as reflection in the display on the remote controller or 
triggering the next associated operation, by transmitting 
a command to the device to be controlled and receiving a 
response (execution result) to the command; 

[0082] 
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(2) Transmission and Reception of Content 

To provide a playback function that is expected as a 
result of control, such as displaying of network content 
(a moving picture or still image) on a screen of the 
remote controller in hand; Further, to realize a control 
function using meta- information of the content; (As an 
example, a channel is selected (a channel is changed) or 
a recording reservation is set using EPG information 
displayed in hand.) 
[0083] 

(3) File Transfer 

To install in the I? remote controller a storage 
device or a slot for a removable medium, such as a memory 
card, and to transfer the recording of the medium at the 
side of the IP remote controller to a content playback 
apparatus on the network to play it back; (As an example, 
a picture stored in a memory medium loaded in the remote 
controller is displayed on a television screen on the 
network . ) 
[0084] 

(4) Remote Operation 

To allow a device that is not in front of the user's 
eyes to be operated if the device is connected to the 
network; (Therefore, content in a recorder located in 
another room or the like can be browsed.) Further, to 
allow the user to connect the remote controller to a home 
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network, while away from home, to perform a similar 
operation to that when the user is at home; 
[0085] 

(5) Integration of Remote Controllers 

To enable mutual communication with a device to be 
controlled. (Therefore, device information is obtained 
from the device to be controlled, thus making it possible 
to switch to a user interface suitable for each device to 
be controlled and making it easy to integrate individual 
remote controllers for devices into a single unit.) 

[0086] 

There are roughly two types of IP remote controllers, 
i.e., a type having only a control function and a type 
having a content playback function in addition to the 
control function. The former type, i.e., an inexpensive 
remote controller, and a highly value-added remote 
controller can individually be constructed. The IP 
remote controller shown in Fig. 3 corresponds to the 
latter type. 
[0087] 

C. Highly Value-Added Function of Remote Control System 
using IP Remote Controller 

In this section, a highly value-added function of the 
remote control system, which is achieved by the basic 
functions discussed in the previous section, will be 
described. 
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[0088] 

C-1. EPG Function 

A so-called electronic program guide (EPG: Electrical 
Program Guide) system is known in which a program guide 
for selecting a television broadcast program is 
superimposed on an image signal to be transmitted and is 
displayed on a display device of a receiver. There are 
EPG systems of the VBI (Vertical Blanking Interyal) type 
and the digital satellite type for use in digital direct 
satellite 'broadcasting (DSS: Digital Satellite System (a 
trademark of Hughes Communication) . 
[0089] 

Either type of EPG system provides an EPG view 
received by a television set on the screen, and allows a 
user to check a desired program on the basis of the EPG 
view to select a channel or set a reservation to record 
the program on the EPG view. A large number of broadcast 
services currently provide EPG. 
[0090] 

In the remote control system according to the present 
embodiment, EPG data received by a television set can be 
used on an IP remote controller to switch television 
channels and further set a recording reservation on the 
EPG view displayed on the LCD 19 of the IP remote 
controller. 
[0091] 
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An operation procedure of the EPG function achieved 
by the remote control system according to the present 
embodiment will be explained with reference to Figs. 5 to 
8. It is assumed that a television set and an HDD 
recorder are provided as devices to be controlled by the 
IP remote controller. Among them, the television set is 
a DLNA device having a network communication function, 
such as ETHERNET™, while the HDD recorder is an IR device 
without a network communication function, which can be 
remotely controlled only by a SIRCS-based infrared 
communication method. It is also assumed that the 
television set is connected to or integrally formed with 
the remote control server (see Fig. 4) and that an 
operation command from the IP remote controller to the 
HDD recorder is received through the network and is 
subjected to protocol conversion by the remote control 
server to transfer the converted command to the HDD 
recorder by means of an infrared communication method. 
It is further assumed that the television receiver and 
the HDD recorder are connected through AV input/output 
terminals. 
[0092] 

Fig. 5 illustrates a manner in which EPG (current) 
data received by the television set is obtained by the IP 
remote controller and is displayed on the LCD 19. 
[0093] 
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On the side of the IP remote controller', Set EPG View 
is entered via the input switch matrix unit 14 . In 
response to this, a request for EPG (current) data is 
transmitted from the IP remote controller to the 
television receiver, which is a DLNA device, through the 
network . 
[0094] 

On the side of the television receiver, the EPG 
(current) data superimposed on the received broadcast 
wave is separated and transferred to the IP remote 
controller through the network. 
[0095] 

The EPG data is written in a structure description 
language, such as an XML (extensible Markup Language) . 
On the side of the IP remote controller, the CPU 11 
analyzes the received EPG (current) data to create an EPG 
view. The GDP 18 performs display processing of an EPG 

(current) view to display and output an EPG view on the 
LCD 19, as illustrated in Fig. 5. 

[0096] 

Fig. 6 illustrates a manner in which a television 
channel change operation is carried out through the EPG 
(current) view displayed on the IP remote controller. 

[0097] 

On the side of the IP remote controller, the user can 
select a channel on the EPG (current) display view by 
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using the input switch matrix unit 14 . When selection of 
a channel is set, a channel change request is transmitted 
to the television receiver through the network. 
[0098] 

The television receiver adjusts a tuner to the 
channel specified in the received channel change request. 
Then, the television receiver notifies the IP remote 
controller of completion of the channel change through 
the network. 
[0099] 

Then, on the side of the IP remote controller, when a 
message indicating the completion of the channel change 
is received, the display screen of the LCD 19 is returned 
to the EPG (current) view. 
[0100] 

Fig. 7 illustrates a manner in which EPG (future) 
data received by the television set is obtained by the IP 
remote controller and is displayed on the LCD 19 . 
[0101] 

On the side of the IP remote controller, Set EPG View 
is entered via the input switch matrix unit 14. In 
response to this, a request for EPG (future) data is 
transmitted from the IP remote controller to the 
television receiver, which is a device to be controlled, 
through the network. 
[0102] 
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On the side of the television receiver, the EPG 
(future) data superimposed on the received broadcast wave 

is separated and transferred to the IP remote controller 

through the network. 
[0103] 

On the side of the IP remote controller, the CPU 11 
analyzes the received EPG (future) data to create an EPG 
view. The GDP 18 performs display processing of an EPG 

(future) view to display and output an EPG view on the 
LCD 19, as illustrated in Fig. 7. 

[0104] 

While the EPG (current) data provides information on 
the program currently broadcasted by each broadcast 
station, the EPG (future) data provides information 
concerning a recording reservation of a program.. Fig. 8 
illustrates a manner in which a recording reservation of 
a program is set through an EPG (future) view displayed 
on the IP remote controller. 
[0105] 

On the side of the IP remote controller, the user can 
select on the EPG (future) display view a television 
program to be reserved for recording by using the input 
switch matrix unit 14 . When selection of a channel is 
set, a request for making a reservation to record the 
selected television program is transmitted to the 
television receiver connected to the remote control 
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server through the network. 

[0106] 

In the illustrated example, the HDD recorder serving 
as a device to be controlled is an IR device that can be 
operated only by an old infrared remote control method. 
For the benefit of the HDD recorder, the remote control 
server converts an operation command received from the IP 
remote controller through the network into a command for 
use in infrared communication such as an SIRCS, and 
performs optical transmission by means of infrared light. 
If the IR device is in a dead angle from the remote 
control server or the distance therebetween is long, an 
infrared command may be output to an AV mouse so that the 
infrared command is optically transmitted in the vicinity 
of the IR device. Meanwhile, if the HDD recorder is near 
the IP remote controller so as to be directly irradiated 
with infrared light, the IP remote controller may 
transmit an SIRCS command for requesting a recording 
reservation from the infrared communication unit 16. 
[0107] 

Fig. 9 shows an operation sequence for displaying the 
EPG (current) data on the IP remote controller. 
[0108] 

The remote control server connected to the television 
receiver stores weekly SI (Service Information) data as 
EPG data. Current SI data superimposed on a received 
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broadcast wave is separated and is stored as EPG data. 

[0109] 

On the side of the IP remote controller, Set EPG View 
is entered via the input switch matrix unit 14 . In 
response to this, a request for EPG (current) data is 
transmitted from the IP remote controller to the 
television receiver, which is a DLNA device, through the 
network . 
[0110] 

The remote control server transfers the EPG (current) 
data to the IP remote controller through the network. On 
the side of the IP remote controller, the CPU 11 analyzes 
the received EPG (current) data to create an EPG view. 
The GDP 18 performs, display processing of an EPG 
(current) view to display and output an EPG view on the 
LCD 19. 
[0111] 

Fig. 10 is a flowchart showing a processing procedure 
executed by the IP remote controller to display the EPG 
(current) data. 
[0112] 

When an EPG (current) data display request is entered 
from the input switch matrix unit 14, first, current time 
is obtained (step SI) . 
[0113] 

Then, a time zone is specified, and an EPG data 
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request is transmitted to the remote control server (step 
S2) . 
[0114] 

When the IP remote controller receives EPG data from 
the remote control server (step S3), the CPU 11 
configures display data of an EPG view based on the EPG 
data (step 34) . 
[0115] 

Then, the GDP 18 outputs the generated display data 
on the screen of the LCD 19 (step 35) . 
[0116] 

By specifying a next time zone instead of the current 
time and submitting an EPG data request, the IP remote 
controller can obtain EPG (future) data, As discussed 
above, a recording reservation of a television program 
can be set through the EPG (future) view. In the 
following, some examples regarding an operation procedure 
for setting a recording reservation of a program in a 
recording device by the IP remote controller using EPG 
(future) data will be described. In either case, it is 
assumed that a recording device, such as an HDD recorder, 
is an IR device without a networlc communication function, 
which can be remotely controlled only by an infrared 
communication method such as 3IRCS. 
[0117] 

Fig. 11 shows an example of an operation sequence in 
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which a recording reservation of a program is set in the 
IR recording device by the IP remote controller using EPG 

(future) data. In the illustrated example, it is assumed 
that the remote control server manages a recording- 
reservation list. 

[0118] 

The remote control server connected to the television 
receiver stores weekly SI data as EPG data. Current SI 
data superimposed on a received broadcast wave is 
separated and stored as EPG data, 
[0119] 

On the side of the IP remote controller, meanwhile, 
it is assumed that an EPG (current) view is currently 
displayed. When Set EPG Next Time Zone View is entered 
via the input switch matrix unit 14, a request for EPG 

(future) data is transmitted in response from the IP 
remote controller to the remote control server integrally 
formed with the television receiver through the network. 

[0120] 

The remote control server transfers the EPG (future) 
'&ata to the IP remote controller through the network. 
Then, on the side of the IP remote controller, the CPU 11 
analyzes the received EPG (future) data to create an EPG 
view. Then, the GDP 18 performs display processing of an 
EPG (future) view to display and output an EPG view on 
the LCD 19. 
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[0121] 

Then, on the side of the IP remote controller, it is 
assumed that the user has selected on the EPG (future) 
display view a television program to be reserved for 
recording by using the input switch matrix unit 14. In 
response to this, a recording reservation request is 
transmitted together with reservation data to the remote 
control server through the network. 
[0122] 

The remote control server analyzes the received 
recording reservation request to determine whether or not 
overlapping reservations which record the same program 
have been requested. If overlapping reservations are 
found, the remote control server returns a reservation- 
overlap status to the IP remote controller through the 
network. On the side of the IP remote controller, an 
error indication is displayed through the LCD 19. 
[0123] 

If there is no overlapping request for making 
recording reservations, the remote control server adds 
the requested recording reservation as a new entry in the 
recording- reservation list managed in the remote control 
server. Then, a completion-of -reservation status is 
returned to the IP remote controller through the network. 
On the side of the IP remote controller, a completion-of - 
reservation indication is displayed through the LCD 19. 
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[0124] 

Subsequently/ the remote control server stands by 
until the reserved recording time. When the reserved 
recording time arrives, the remote control server 
transmits an SIRCS command for turning on the power 
supply to the IR recording device to start the recording 
of the television program reserved for recording. 
[0125] 

Then, when the record-setting time has elapsed, the 
remote control server transmits an SIRCS command for 
stopping the recording operation and then a SIRCS command 
for turning off the power supply to the IR recording 
device to terminate the overall recording reservation 
operation. 
[0126] 

In the example operation shown in Fig. 11, it is 
necessary to render both the remote control server and 
the IR recording device to be active at the reserved 
recording time. 
[0127] 

Fig. 12 shows another example of the operation 
sequence in which a recording reservation of a program is 
set in the recording device by the IP remote controller 
using EPG (future) data. In the illustrated example, it 
is assumed that the remote control server manages a 
recording-reservation list and transfers reservation data 
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to the recording device. 
[0128] 

The remote control server connected to the television 
receiver stores weekly SI data as EPG data. Current SI 
data superimposed on a received broadcast wave is 
separated and stored as EPG data. 
[0129] 

On the side of the IP remote controller, meanwhile, 
it is assumed that an EPG (current) view is currently 
displayed. When Set EPG Next Time Zone View is entered 
via the input switch matrix Tinit 14, a request for EPG 

(future) data is transmitted in response from the IP 
remote controller to the remote control server integrally 
formed with the television receiver through the network. 

[0130] 

The remote control server transfers the EPG (future) 
data to the IP remote controller through the network. 
Then, on the side of the IP remote controller, the CPU 11 
analyzes the received EPG (future) data to create an EPG 
view. Then, the GDP 18 performs display processing of an 
EPG (future) view to display and output an EPG view on 
the LCD 19. 
[0131] 

Then, on the side of the IP remote controller, it is 
assumed that the user has selected on the EPG (future) 
display view a television program to be reserved for 
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recording by using the input switch matrix unit 14. In 
response to this, a recording reservation request is 
transmitted together with reservation data to the remote 
control server through the network. 
[0132] 

The remote control server analyzes the received 
recording reservation request to determine whether or not 
overlapping reservations which record the same program 
have been requested. If overlapping reservations are 
found, the remote control server returns a reservation- 
overlap status to the IP remote controller through the 
network. On the side of the IP remote controller, an 
error indication is displayed through the LCD 19 . 
[0133] 

If there is no overlapping request for making 
recording reservations, the remote control server adds 
the requested recording reservation as a new entry in the 
recording- reservation list managed in the remote control 
server. Then, a completion-of -reservation status is 
returned to the IP remote controller through the network. 
On the side of the IP remote controller, a completion-of - 
reservation indication is displayed through the LCD 19. 
[0134] 

If the recording device is an IR device that can be 
remotely controlled only by an infrared communication 
method, the recording-reservation request command 
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transferred through the IP network is subjected to IP/IR 
protocol conversion to produce an infrared communication 
command, such as SIRCS, to transfer the reservation data 
to the recording device. The reservation data includes 
meta- information obtained from the EPG, such as the name 
of a television program to be reserved for recording, the 
recording date and time, and the broadcast station. 
[0135] 

Upon receiving recording-reservation data, the 
recording device adds the requested recording reservation 
as a new entry in the recording- reservation list managed 
in the recording device. When the reserved recording 
time arrives, the power supply of the recording device is 
turned on to start the recording of the television 
program reserved for recording. 
[0136] 

Then, when the record-setting time has elapsed, the 
recording device stops the recording operation, and turns 
off the power supply to terminate the overall recording 
reservation operation. 
[0137] 

In the example operation shown in Fig. 12, it is 
sufficient to render only the recording device to be 
active at the reserved recording time, and the power 
supply of the remote control server can be turned off 
after completion of the recording reservation. 
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[0138] 

Fig. 13 shows still another example of the operation 
sequence in which a recording reservation of a program is 
set in the recording device by the IP remote controller 
using EPG (future) data. In the illustrated example, it 
is assumed that the recording device manages a recording- 
reservation list. 
[0139] 

The remote control server connected to the television 
receiver stores weekly SI data as EPG data. Current SI 
data superimposed on a received broadcast wave is 
separated and stored as EPG data. 
[0140] 

On the side of the IP remote controller, meanwhile, 
it is assumed that an EPG (current) view is currently 
displayed. When Set EPG Next Time Zone View is entered 
via the input switch matrix unit 14, a request for EPG 
(future) data is transmitted in response from the IP 
remote controller to the remote control server integrally 
formed with the television receiver through the network. 
[0141] 

The remote control server transfers the EPG (future) 
data to the IP remote controller through the network. 
Then, on the side of the IP remote controller, the CPU 11 
analyzes the received EPG (future) data to create an EPG 
view. Then, the GDP 18 performs display processing of an 
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EPG (future) view to display and output an EPG view on 
the LCD 19. 
[0142] 

Then, on the side of the IP remote controller, it is 
assumed that the user has selected on the EPG (future) 
display view a television program to be reserved for 
recording by using the input switch matrix unit 14. In 
response to this, a recording reservation request is 
transmitted together with reservation data to the remote 
control server through the network. 
[0143] 

If the recording device is an IR device that can be 
remotely controlled only by an infrared communication 
method, the remote control server performs IP/IR protocol 
conversion on the recording- reservation request command 
transferred through the IP network to produce an infrared 
communication command, such as SIRCS, and transfers the 
reservation data to the recording device. The 
reservation data includes meta- information obtained from 
the EPG, such as the name of the television program to be 
reserved for recording, the recording date and time, and 
the broadcast station. The remote control server returns 
a status indicating completion of the transfer of the 
recording- reservation request to the IP remote controller 
through the network. 
[0144] 
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On the side of the recording device, upon receiving 
reservation data via infrared communication, it is 
analyzed to determine whether or not overlapping 
reservations to record the same program have been 
requested. If overlapping reservations are found, an 
error indication is displayed. If there is no 
overlapping request for making recording reservations, 
the requested recording reservation is added as a new 
entry in the recording- reservation list managed in the 
recording device, and a completion-of -reservation 
indication is displayed. 
[0145] 

When the reserved recording time arrives, the 
recording device turns on the power supply to start the 
recording of the program reserved for recording. 
[0146] 

Then, when the record- setting time has elapsed, the 
recording device stops the recording operation, and then 
turns off the power supply to terminate the overall 
recording reservation operation. 
[0147] 

C-2. Moving-Image Playback Function 

As discussed above, the IP remote controller 
according to the present embodiment has a codec 
processing function and a graphic display function, and 
is capable of playing back a moving image. For example, 
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the IP remote controller decodes AV content distributed 
from a television receiver, a remote control server, or 
the like, which is a device to be controlled, to output 
audio and video. 
[0148] 

In this connection, the remote control server or the 
like encodes an audio signal and video signal received as 
analog broadcast waves into a data format that can be 
played back on the IP remote controller, such as MPEG4 
data, to distribute the encoded data to the IP remote 
controller through the IP network. Alternatively, the 
remote control server loads recorded content recorded at 
a high bit rate, such as MPEGl or MPEG2 content, from an 
HDD recorder through the LAN, converts it into a low-bit - 
rate data format that can be received and played back by 
the IP remote controller, such as MPEG4 data, and 
distributes the converted data via streaming through the 
LAN. 
[0149] 

An operation procedure of the moving- image playback 
function achieved by the remote control system according 
to the present embodiment will be explained with 
reference to Figs. 14 to 19. It is assumed that a 
television set and an HDD recorder are provided as 
devices to be controlled by the IP remote controller. 
Among them, the television set is a DLNA device having a 
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network communication function, such as ETHERNET™, while 
the HDD recorder is an IR device without a network 
communication function, which can be remotely operated 
only by a SIRCS-based infrared communication method. It 
is also assumed that the television set is connected to 
or integrally formed with the remote control server (see 
Fig. 4) and that an operation command from the IP remote 
controller to the HDD recorder is received through the 
network and is subjected to protocol conversion by the 
remote control server to transfer the converted command 
to the HDD recorder by means of an infrared communication 
method. Further, the remote control server and the HDD 
recorder are connected through AV input/output terminals. 
[0150] 

Fig. 14 illustrates a manner in which a terrestrial 
analog broadcast received by the television set is viewed 
by using the IP remote controller. 
[0151] 

On the side of the IP remote controller, the 
television set is selected as a device to be operated 
through the input switch matrix unit 14 . A request for 
turning on the power supply is transmitted from the IP 
remote controller to the television receiver serving as 
the device to be controlled, and the power supply of the 
television set is turned on in response to the request. 
[0152] 
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Then, on the side of the IP remote controller, a 
channel is selected through the input switch matrix unit 
14. When a desired channel is determined, television 
viewing is set. Then, a moving-image transfer request is 
transmitted from the IP remote controller to the 
television set. 
[0153] 

On the side of the television set, in response to the 
moving- image transfer request, an analog broadcast wave 
of the channel in which the viewing is set is received. 
Then, the analog video and audio signals are converted 
into a data format that can be handled by the IP remote 
controller, such as MPEG4 data, and are then distributed 
via streaming to the IP remote controller. 
[0154] 

The IP remote controller decodes the thus transferred 
moving- image data to display the television program on 
the LCD 19. 
[0155] 

The operation of displaying a television program 
currently being viewed on the television set or streaming 
video from the recording device on the IP remote 
controller is also referred to herein as a "catch" . 
[0156] 

Figs. 15 to 16 illustrate a manner in which a channel 
is changed in a period of time during which a terrestrial 
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analog broadcast received by the television set is viewed 
on the IP remote controller. It is assumed that the 
television set has a multi- tuner function capable of 
receiving broadcast waves from two or more channels at 
the same time. 
[0157] 

As discussed above, the television set encodes a 
received broadcast program and distributes it to the IP 
remote controller via streaming, and the IP remote 
controller receives and decodes it to display the 
television program on the LCD 19. At this time, the same 
program is being displayed on the television screen and 
the LCD 19 of the IP remote controller. 
[0158] 

On the side of the IP remote controller, a channel is 
selected through the input switch matrix unit 14. For 
example, if the IP remote controller is provided with a 
user interface, an operation for selecting the television 
set as the medium and placing a cursor position onto a 
desired channel is performed (not shown) . 
[0159] 

When a channel change is set, a channel -change 
request is transmitted from the IP remote controller to 
the television set. 
[0160] 

In response to the channel -change request, the 
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television set receives an analog broadcast wave on the 
determined channel, converts it into a data format that 
can be handled by the IP remote controller, such as MPEG4 
data, and distributes the converted data to the IP remote 
controller via streaming. 
[0161] 

Since the channel change is carried out on the IP 
remote controller, the television set still displays the 
program received on the initial channel. On the side of 
the IP remote controller, meanwhile, since the 
transferred moving- image data is decoded and displayed on 
the LCD 19, the television program of the changed channel 
can be viewed. 
[0162] 

Fig. 17 illustrates a manner in which the video 
displayed on the television set is changed to the 
television program currently being viewed on the IP 
remote controller. 
[0163] 

As shown in Fig. 17, a screen change request for 
changing to the channel currently being viewed on the IP 
remote controller is transmitted to the television set. 
[0164] 

In response to this request, the television set 
changes the screen to the designated channel, and 
displays the video of the television program of the 
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changed channel. Then, the currently received analog 
video and audio signals are converted into a data format 
that can be handled by the IP remote controller, such as 
MPEG4 data, and is distributed to the IP remote 
controller via streaming. 
[0165] 

The IP remote controller decodes the thus transferred 
moving- image data to display the television program on 
the LCD 19. As a result, the same program is displayed 
on the television screen and the LCD 19 of the IP remote 
controller. 
[0166] 

The operation of displaying a television program or 
streaming video currently being viewed on the IP remote 
controller on the television set is also referred to 
herein as a "throw". 
[0167] 

Figs. 18 to 19 illustrate a manner in which recorded 
content stored in the recording device is viewed on the 
IP remote controller. 
[0168] 

On the side of the IP remote controller, the user 
uses the input switch matrix unit 14 to give instructions 
to turn on the recording device, such as an HDD recorder, 
and to distribute recorded content. These requests are 
transmitted to the remote control server integrally 
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formed with the television receiver through the network. 
[0169] 

In the illustrated example, the HDD recorder, which 
is the device to be controlled, is an IR device that can 
be operated only by an infrared remote control method. 
For the benefit of the HDD recorder, the remote control 
server performs IP/IR conversion on an operation command 
received from the IP remote controller through the 
network into a command for use in infrared communication 
such as SIRCS, and performs optical transmission by means 
of infrared light. If the legacy controlled device is in 
a dead angle from the remote control server or the 
distance therebetween is long, an infrared command is 
output to an AV mouse so that the infrared command is 
optically transmitted in the vicinity of the IR device. 
Meanwhile, if the HDD recorder is near the IP remote 
controller so as to be directly irradiated with infrared 
light, the IP remote controller may output. an SIRCS 
command for requesting a recording reservation from the 
infrared communication unit 16. 
[0170] 

In response to the SIRCS command, the HDD recorder 
turns on the power supply thereof, and supplies AV output 
video of the designated recorded content to the remote 
control server, 
[0171] 
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The remote control server encodes the AV output video 
of the HDD recorder into a data format that can be played 
back by the IP remote controller, such as MPEG4 data, and 
distributes it to the IP remote controller through the IP 
network. Alternatively-, the remote control server loads 
recorded content recorded at a high bit rate, such as 
MPEGl or MPEG2 content, from the HDD recorder, converts 
it into a low-bit -rate data format that can be received 
and played back by the IP remote controller, such as 
MPEG4 data, and distributes the converted data via 
streaming through the LAN. 
[0172] 

On the side of the IP remote controller, the 
transferred moving- image data is decoded and displayed on 
the LCD 19, thus allowing the user to view the recorded 
content. The television set, on the other hand, still 
displays the program received on the initial channel. 
[0173] 

As shown in Fig. 19, it is also possible to change 
the channel on the television side to the recorded 
content currently being viewed on the IP remote 
controller by means of a throw operation (discussed 
above) . 
[0174] 

In this case, a screen change request for changing to 
the recorded content currently being viewed on the IP 
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remote controller is transmitted to the television set. 
[0175] 

On the side of the television set, in response to 
this request, the AV output video from the HDD recorder 
is encoded and distributed to the IP remote controller 
while the display screen is also changed from the 
television video to the AV output video. 
[0176] 

Fig. 20 shows an example of an operation sequence for 
distributing AV content via streaming from an IR 
recording device that can be remotely controlled only by 
an infrared communication method to the IP remote 
controller. 
[0177] 

In this case, a remote control server having a 
networlc connection function and a protocol conversion 
function into an infrared command is provided. The 
remote control server is connected to or integrally 
formed with, for example, a television set. Further, it 
is assumed that the remote control server pre-registers 
therein information concerning IR devices to which an 
infrared command can be transmitted from the remote 
control server, 
[0178] 

The IP remote controller, when turned on, submits a 
request for device information concerning the IR devices 
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to the remote control server through the network. 
[0179] 

In response to the request for the device information, 
the remote control server returns the information 
concerning the IR devices to the IP remote controller 
through the network. 
[0180] 

The received device list is displayed on the LCD 19 
of the IP remote controller. When the user selects an IR 
device on this screen, a request for starting streaming 
of AV content to the selected IR device is transmitted to 
the remote control server through the network. 
[0181] 

Upon receiving the request for starting streaming, 
the remote control server determines whether or not the 
requested IR device has been turned on and is outputting 
video. If the IR device has not been turned on, the IR 
device is turned on to start outputting video. 
[0182] 

Then, the remote control server encodes the AV output 
video from the IR device into a data format that can be 
played back by the IP remote controller, such as MPEG4 
data, and distributes it to the IP remote controller via 
streaming through the IP network. 
[0183] 

The IP remote controller decodes the thus transferred 
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moving- image data to display video on the LCD 19. 
[0184] 

On the side of the IP remote controller, furthermore, 
even during the streaming display, operations for the IR 
device can be performed through the input switch matrix 
unit 14 . The operations used herein include changing of 
output video, changing of channels, and so forth. 
[0185] 

A command for the IR device is transmitted from the 
IP remote controller to the remote control server through 
the networJc. The remote control server converts the 
operation command received from the IP remote controller 
through the networJc into a command for use in infrared 
communication such as SIRCS, and performs optical 
transmission by means of infrared light. 
[0186] 

The IR device executes the command received via 
infrared light. Then, streaming of video in which the 
execution result of the command has been reflected is 
performed via the remote control server, 
[0187] 

If tlie device to be controlled by the IP remote 
controller is an IR device that can be remotely 
controlled only by an infrared communication method, as 
discussed above, a remote control server for performing 
protocol conversion on a command is provided. If the 
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device to be controlled is a DLNA device having a network 
connection function, meanwhile, the intervention of the 
remote control server is not necessary because the IP 
remote controller is capable of performing a command 
operation through the network. 
[0188] 

Fig. 21 shows an example of an operation sequence for 
streaming AV content from a DLNA device to the IP remote 
controller. 
[0189] 

The IP remote controller, when turned on, submits a 
request for device information to the DLNA device through 
the network. 
[0190] 

In response to the device information request, the 
DLNA device, which has been turned on, returns the device 
information to the IP remote controller through the 
network. 
[0191] 

A device list concerning DLNA devices that have 
responded is displayed on the screen of the LCD 19 of the 
IP remote controller. A DLNA device that responded with 
device information in the past is displayed as grayed-out. 
A DLNA device that has not responded for a predetermined 
period of time is determined to be disconnected from the 
network and is deleted from the device list. 

-65- 



Application No.: 10/589,992 Attorney Docket No.: SON-3470 

Clean Substitute Specification 

[0192] 

When the user selects a DLNA device on the screen, it 
is determined whether or not the selected DLNA device has 
been turned on, that is, whether or not the device 
information has been returned. If the DLNA device has 
not been turned on, a request for turning on the power 
supply is transmitted to the DLNA device through the 
network . 
[0193] 

Upon receiving the request for turning on the power 
supply, the DLNA device turns on the power supply, and 
returns a network participation notification to the IP 
remote controller. The IP remote controller reflects the 
activation of the DLNA device in the device list, and 
changes the grayed-out indication of the DLNA device to a 
normal indication . 
[0194] 

If the selected DLNA device has been turned on, a 
request for a list of available content is submitted to 
the selected DLNA device through the network. In 
response to this, the DLNA device returns a content list 
to the IP remote controller. For example, the content 
list is a list of recorded content if the DLNA device is 
a recording device, such as an HDD recorder, and is a 
list of receivable channels if the DLNA device is a 
television set. 

-66- 



Application No.: 10/589,992 Attorney Docket No.: SON-3470 

Clean Substitute Specification 

[0195] 

The received content list is displayed on the screen 
of the LCD 19 of the IP remote controller. When the user 
selects desired content on the screen, a request for 
starting streaming of the selected content is transmitted 
to the remote control server through the network. 
[0196] 

Upon receiving the request for starting streaming, 
the DLNA device outputs the corresponding video, and 
encodes the AV output video into a data format that can 
be played back by the IP remote controller, such as MPEG4 
data, to distribute the encoded data to the IP remote 
controller via streaming through the IP network. 
[0197] 

The IP remote controller decodes the thus transferred 
moving-image data to display video on the LCD 19. 
[0198] 

On the side of the IP remote controller, furthermore, 
even during the streaming display, operations for the 
content can be performed through the input switch matrix 
unit 14. The operations for the content used herein 
include changing of output video, changing of channels, 
and so forth. 
[0199] 

A command for the content operations is sent from the 
IP remote controller to the DLNA device through the 
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network. The DLNA device executes the received command 
to perform streaming of video in which the execution 
result of the command has been reflected. 
[0200] 

In accordance with the operation sequence shown in 
Fig. 20, the video content output from the IR recording 
device can be viewed on the IP remote controller. Fig. 
22 shows an example of an operation sequence for 
"throwing" content of an IR device, currently being viewed 
on the IP remote controller onto a television set. It is 
assumed that the television set has a remote control 
server function. 
[0201] 

AV video is output from an IR recording device, such 
as an HDD recorder or any other recording device. The 
television set utilizes the remote control server 
function to encode the AV output video into a data format 
that can be played back by the IP remote controller, such 
as MPEG4 data, to distribute the encoded data to the IP 
remote controller via streaming through the IP network. 
The IP remote controller decodes the transferred moving- 
image data to display video on the LCD 19 . 
[0202] 

It is assumed that a throw operation has been 
performed through the input switch matrix unit 14 at the 
side of the IP remote controller. The IP remote 
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controller instructs the television set to stop streaming, 
and the television set stops streaming to the IP remote 
controller in response to the instruction. 
[0203] 

Then, the IP remote controller instructs the 
television set to play back the content of the IR device. 
At this time, content is specified and played back. 
There are several sets by which the content is specified, 
such as a device, which is a content provider, and a 
display destination, the name of an AV input terminal of 
the television set and a display destination, and a 
function name of the television set and a display 
destination. 
[0204] 

On the side of the television set, in response to the 
content playback instruction, AV output video from the IR 
recording device is displayed and output. 
[0205] 

In accordance with the operation sequence shown in 
Fig. 21, video content output from the DLNA device can 
directly be viewed on the IP remote controller. Fig. 23 
shows an example of an operation sequence for "throwing" 
content of a DLNA device currently being viewed on the IP 
remote controller onto a television set. 
[0206] 

An HDD recorder or any other DLNA device having a 
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recording function encodes AV output video into a data 
format that can be played by the IP remote controller, 
such as MPEG4 data, and distributes the encoded data to 
the IP remote controller via streaming through the IP 
network. The IP remote controller decodes the 
transferred moving- image data to display video on the LCD 
19. 
[0207] 

It is assumed that a throw operation has been 
performed through the input switch matrix unit 14 at the 
side of the IP remote controller. The IP remote 
controller instructs the DLNA device to stop streaming to 
the IP remote controller. In response to the instruction, 
the DLNA device stops streaming to the IP remote 
controller. 
[0208] 

Then, the IP remote controller submits a content 
playback instruction through the network. In the 
playback instruction, a server, a content name, a 
playback start point, and a display destination are 

specified. 

[0209] 

In response to this, the television set specified as 
the destination onto which the content is thrown requests - 
the DLNA device to perform streaming. The DLNA device 
starts streaming of the specified content from the 
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specified playback start point. The television set 
displays and outputs the received streaming video. 
Industrial Applicability 
[0210] 

The present invention has been described in detail 
with reference to specific embodiments. However, it is 
obvious that modifications and alternatives of the 
embodiments may be made by those skilled in the art 
without departing from the scope of the present invention. 
[0211] 

While a device to be controlled by an IP remote 
controller has been described herein in the context of a 
television receiver or a recording device such as an HDD 
recorder, the present invention is not limited thereto. 
Other devices handling AV content, or home electric 
appliances or information devices that do not handle AV 
content may also be used as devices to be controlled by 
the IP remote controller to construct the remote control 
system according to the present invention. 
[0212] 

That is, the present invention has been disclosed by 
way of exemplary embodiments, and the contents disclosed 
herein should not be restrictively construed. The gist 
of the present invention should be determined in 
consideration of the claims. 
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